The Shortest History of AI by Toby Walsh

Author:Toby Walsh
Language: eng
Format: epub
Publisher: The Experiment

By 2009, ImageNet had over three million images and was released to the world. It has since grown to over fourteen million images, all helpfully labeled into one of twenty-one thousand categories such as “balloon,” “banana” or “boat.” And then, to promote ImageNet in particular and AI research in general, an annual competition called the ImageNet Large Scale Visual Recognition Challenge began in 2010. This pitted the best computer vision algorithms in the world against each other. The competition was a great success, focusing the research community’s attention and accelerating progress in AI.

In the 2012 ImageNet competition, Geoff Hinton, Alex Krizhevsky and Ilya Sutskever, another of Hinton’s PhD students, put all three ingredients for deep learning together for the first time: backpropagation over a deep network, computation using GPUs and lots of training data. Their simple recipe worked a treat; indeed, their entry blew the opposition out of the water. The eight-layer deep learning network they built, AlexNet, had an error rate of just 15.3 percent, beating the runner-up by more than 10.8 percentage points. This remains the largest winning margin in the competition’s history.

Interestingly, AlexNet wasn’t the first deep neural network, the first GPU-powered neural network, or even the first GPU-powered deep neural network to win a competition by a large margin. Yann LeCun is now chief AI scientist at Meta and a professor at New York University. But back in 1995, he and some of his colleagues at Bell Labs in New Jersey had a pioneering seven-layer neural network called LeNet-5. This was deployed by companies like NCR to read over 10 percent of checks in the United States during the late 1990s and early 2000s. It was also used by the US Postal Service to help sort letters by recognizing the digits in zip codes on handwritten letters. Various other researchers had also used GPUs to speed up neural networks since at least 2006. And Jürgen Schmidhuber’s DanNet, a GPU-powered deep neural network, had already won four computer vision competitions in a row in 2011 and 2012, in some cases by impressive margins.3

But it was AlexNet that put all three ingredients together and caught the attention of AI researchers around the world. It announced, with a big bang, the start of the deep learning revolution. And the echo of that bang can still be heard today in AI systems like ChatGPT from OpenAI and Gemini from Google.

In the Northern-Hemisphere autumn of 2012, Hinton, Krizhevsky and Sutskever founded a start-up company to capitalize on their recipe for deep neural network research called DNNResearch. Just a few months later, in December 2012, at the main conference for neural network research at Harrah’s and Harvey’s Casino on Lake Tahoe, Hinton decided to cash out. He organized an auction of DNNResearch, inviting Google, Microsoft, Baidu and DeepMind (which was yet to be bought by Google) to bid. Hinton stopped the auction when Google bid forty-four million dollars. He could have undoubtedly got more, but this was, he felt, a fair price

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.